Code conversion method and device thereof

ABSTRACT

In order to cope with format conversion of user data, the bit rate value and VBV (Video Buffering Verifier) buffer size value in a sequence header of an input code and the VBV delay value in a picture header of the input code are modified to obtain an intermediate code ( 305  to  309 ), and additional information ( 300 ) is generated for distinguishing GOP (Group of Pictures) user data ( 307 ) from the other main data. A VBV buffer simulation is performed using this additional information ( 300 ) to multiplex the GOP user data in a picture user data region to a data amount such that the operation does not fail, whereby an output code is generated.

TECHNICAL FIELD

The present invention relates to a code conversion method or a codetranslation method for compressed/encoded multimedia information and adevice thereof. Specifically, the present invention relates to formatconversion and addition of user data.

BACKGROUND ART

ISO 13818-2 has been known as one of the standards relating tocompression/encoding techniques for multimedia information which aregenerally called as MPEG-2 (Moving Picture Experts Group Phase 2). Avideo stream of MPEG-2 has a hierarchical structure which includes asequence layer, a GOP (Group of Pictures) layer, a picture layer, aslice layer, a macroblock layer, and a block layer in this order fromthe uppermost layer. Each layer has a special pattern of 4-byte lengthcalled a start code at the leading end thereof. This start code startswith three bytes of 0x00, 0x00, and 0x01, and the following one byteindicates the type of data subsequent to the start code. In thisspecification, “0x” means hexadecimal representation. For example, thefourth bytes of the start codes of the sequence layer, the GOP layer,the picture layer, and the slice layer are 0xB3, 0xB8, 0x00, and 0xAF,respectively.

In MPEG-2, setting of a user extension area is allowed in each of thesequence layer, the GOP layer, and the picture layer. Although it isprescribed that the start code of user data is formed by four bytes of0x00, 0x00, 0x01, and 0xB2, user data of an arbitrary format determinedbased on unique user extension can be placed in any user extension area.

Actually, in each of digital storage media, such as a DVD (DigitalVersatile Disk), and the like, and digital broadcasting, such as DVB(Digital Video Broadcasting), and the like, unique user extension isrealized in compliance with MPEG-2. That fact is that, for example, thestandards of user data for closed caption are not unified, and formatconversion between user data is necessary.

A realtime encoding technique for storing text information, such asEnglish subtitle information, or the like, in a GOP header as user datahas been known (see Japanese Laid-Open Patent Publication No.2001-145067).

A bit rate converter (transcoder) with which the process amount inconversion between TSs (Transport Streams) can be reduced has beenproposed (see Japanese Laid-Open Patent Publication No. 2001-251616).

As a matter of course, the simplest method for converting the format ofuser data is to connect a decoder of a system and an encoder of anothersystem. However, unnecessary processes are performed on data other thanthe user data, i.e., main data, and can be a cause of deterioration inimage quality.

Nevertheless, if only the format of user data included in an input codeis simply converted to obtain an output code, the data amount greatlychanges because of the conversion. Accordingly, there is a possibilitythat rate control fails. As parameters for determining the allowablerange of data amount, a bit rate value and a VBV (Video BufferingVerifier) buffer size value are included in a sequence header, and a VBVdelay value is included in a picture header. There is a possibility thatrate control based on these parameters fails. The same applies to a casewhere user data is added to an input code to generate an output code.

DISCLOSURE OF INVENTION

An objective of the present invention is to provide a code translationmethod, and a device thereof, with which format conversion or additionof user data is realized without failure of rate control.

In order to achieve this objective, according to the present invention,an input code stored in a hierarchical data structure is converted togenerate an output code. The input code includes (1) a parameter relatedto the amount of data encoded by the input code, (2) user data at afirst level of the hierarchical data structure, (3) and main data at asecond level of the hierarchical data structure. The output code, storedin the hierarchical data structure, is generated by modifying the inputcode by moving the user data to a third level of the hierarchical datastructure, and changing the parameter to reflect the change in code sizeeffected by the moving of the user data. Further, the main data includedin the output code is identical to the main data included in the inputcode.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing an example of a structure of a codetranslation device according to the present invention.

FIG. 2, FIG. 3 and FIG. 4 are data format diagrams of data used in thecode translation device of FIG. 1. FIG. 2 illustrates the format of aninput code. FIG. 3 illustrates the format of an intermediate code. FIG.2 illustrates the format of an output code.

FIG. 5 is a block diagram showing an example of an internal structure ofa data analyzing section shown in FIG. 1.

FIG. 6 is a block diagram showing an example of an internal structure ofa multiplexing section shown in FIG. 1.

FIG. 7 is a block diagram showing an example of an internal structure ofa main data reprocessing section shown in FIG. 6.

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, an embodiment of the present invention which is directed tocode translation is described in detail with reference to the drawings.It should be noted that a problem is simplified. It is assumed that, inthe first place, an input code is a video stream of MPEG-2 and istranslated to an output code of a predetermined format. The input codehas a user extension area of GOP layer (GOP user data area), and theoutput code has user extension areas of picture layer (picture user dataareas). For example, user data for closed caption is placed in each ofthese areas. It is further assumed that one GOP includes 15 frames atthe maximum, and only user data corresponding to some leading frames areincluded in one GOP. That is, it is assumed that one GOP does notnecessarily include all of user data corresponding to every picture.Before translation, only one out of 15 frames is provided with a 4-bytestart code. After translation, every frame is provided with a 4-bytestart code of user data. Thus, even when only the start codes areconsidered on the assumption that there are 30 frames per second, thebit rate is increased by translation by (14/15)×30×4×8 bps. It isfurther assumed that the position of user data after translation is notstrictly determined. For example, user data for closed caption is notrequired to have a perfect synchronization with picture data.

The above-described assumption about the user data position and acondition where there are not necessarily user data corresponding to allof the frames are appropriate assumptions in existing systems for closedcaption, and the like.

FIG. 1 shows an example of a structure of a code translation deviceaccording to the present invention. The code translation device of FIG.1 includes a data analyzing section 101, a data buffer 102, and amultiplexing section 103. For example, the data analyzing section 101and the multiplexing section 103 constitute a single LSI called a streamcontroller 104. An external memory which functions as the data buffer102 is connected to the LSI. The data analyzing section 101 has afunction of analyzing an input code 121 to change parameters whichdetermine the allowable range of data amount in the input code 121 (bitrate value, VBV buffer size value, and VBV delay value) so as to complywith the format conversion of user data, and generate additionalinformation which is used for distinguishing the user data included inthe input code 121 from the other main data. The data buffer 102 is amemory for temporarily storing an input code which has changedparameters together with the additional information. Reference numeral122 denotes a write address which is supplied from the data analyzingsection 101 to the data buffer 102. Reference numeral 123 denotes writedata which is supplied from the data analyzing section 101 to the databuffer 102. Reference numeral 124 denotes the write data size aboutwhich the multiplexing section 103 is notified by the data analyzingsection 101. The multiplexing section 103 has a function of multiplexingan input code which has changed parameters and user data in apredetermined format according to the additional information stored inthe data buffer 102 to generate an output code 127 according to thechanged parameters. Reference numeral 125 denotes a read address whichis supplied from the multiplexing section 103 to the data buffer 102.Reference numeral 126 denotes read data which is supplied from the databuffer 102 to the multiplexing section 103.

FIG. 2 is a data format diagram showing the input code 121. FIG. 3 is adata format diagram showing an intermediate code stored in the databuffer 102. FIG. 4 is a data format diagram showing the output code 127.

In FIG. 2, reference numeral 201 denotes a sequence header, referencenumeral 202 denotes a GOP header, reference numeral 203 denotes GOP userdata, reference numeral 204 denotes a picture header, and referencenumeral 205 denotes picture data. The region of the GOP user data 203includes user data for closed caption. FIG. 2 shows only one picturebut, in actuality, the unit of “picture header 204+picture data 205” isrepeated a number of times which corresponds to the number of pictures.

In the intermediate code shown in FIG. 3, reference numeral 305 denotesa sequence header, reference numeral 306 denotes a GOP header, referencenumeral 307 denotes GOP user data, reference numeral 308 denotes apicture header, and reference numeral 309 denotes picture data. Theserespectively correspond to the sequence header 201, the GOP header 202,the GOP user data 203, the picture header 204, and the picture data 205shown in FIG. 2. This intermediate code further includes additionalinformation 300 for distinguishing the GOP user data 307 from the maindata, such as the picture data 309, and the like. In the additionalinformation 300, reference numeral 301 denotes picture information whichincludes the picture size and picture type, reference numeral 302denotes a user data position, reference numeral 303 denotes a user datasize, and reference numeral 304 denotes a picture data position. In thepicture information 301, the picture size represents the entire size ofa picture included in one processing unit, and the picture typerepresents the coding type of I (Intra) picture, P (Predictive) picture,or B (Bidirectionally predictive) picture. The leading position and sizeof the GOP user data 307 are represented by the user data position 302and the user data size 303, respectively. The leading position of thepicture data 309 is represented by the picture data position 304.

The data analyzing section 101 stores data having the additionalinformation 300 shown in FIG. 3 in the data buffer 102. Since a startcode, which is a special pattern of 4-byte length, is always insertedbetween data, such a data structure can be readily created in the caseof a randomly-accessible memory. Further, by providing the additionalinformation 300, a data buffer access with the GOP user data 307 beingdistinguished from the other data is readily realized. Furthermore,since there is the picture information 301 which includes the picturesize, the end of the data structure of FIG. 3, i.e., the starting pointof the subsequent additional information 300, can readily be accessed.

The bit rate value and VBV buffer size value define the upper limit ofthe data amount. The VBV delay value defines the lower limit of the dataamount in the case of CBR (Constant Bit Rate). However, when the VBVdelay value is 0xffff, it is regarded as VBR (Variable Bit Rate), andthe restriction on the lower limit of the data amount is removed.

The bit rate value and VBV buffer size value included in the sequenceheader 305 and the VBV delay value included in the picture header 308are already rewritten by the data analyzing section 101 so as to complywith format conversion of user data. For example, the bit rate value isincreased by (14/15)×30×4×8 bps, which is an estimated value of a bitrate increase by format conversion of user data. The VBV buffer sizevalue is changed to the maximum value which is allowed by the standards.The VBV delay value is set to 0xffff, whereby the setting of VBR isprovided. If the user data is reduced by format conversion, therewritten bit rate value may be smaller than the original bit ratevalue. The rewritten VBV buffer size value only needs to be greater thanan estimated maximum picture size. Alternatively, the original VBVbuffer size value may be used as it is. A method can also be usedwherein the setting of VBR is abandoned and the VBV delay value isrecalculated still at CBR to perform stuffing. Alternatively, theseparameter changes may be performed in the multiplexing section 103.

The multiplexing section 103 multiplexes the GOP user data 307 with theother main data according to the additional information 300 included inthe data buffer 102 to generate an output code 127 which has the formatshown in FIG. 4.

In FIG. 4, reference numeral 401 denotes a sequence header, referencenumeral 402 denotes a GOP header, reference numeral 403 denotes apicture header, reference numeral 404 denotes picture user data, andreference numeral 405 denotes picture data. These respectivelycorrespond to the sequence header 305, the GOP header 306, the pictureheader 308, the GOP user data 307, and the picture data 309 shown inFIG. 3. FIG. 4 shows only one picture but, in actuality, the unit of“picture header 403+picture user data 404+picture data 405” is repeateda number of times which corresponds to the number of pictures.

Hereinafter, details of the data analyzing section 101 and themultiplexing section 103 are described with reference to FIGS. 5 and 6.

FIG. 5 shows an example of an internal structure of the data analyzingsection 101 shown in FIG. 1. The data analyzing section 101 of FIG. 5includes first, second, third and fourth input registers 501, 502, 503and 504, a start code detection section 505, a data analysis controlsection 506 for controlling the entire operation of the data analyzingsection 101, and a selector 507 which is used for insertion of theadditional information 300.

The data analyzing section 101 repeats the following steps 1 to 7 forevery single byte. Specifically, data is taken in at step 1. At step 2,it is determined whether or not the data and the previously-input 3-bytedata constitute a start code. If a predetermined condition is satisfiedat step 2, the process of writing additional information is performed atstep 3. After this process, a pointer for data writing is increased by apredetermined size as a preparation for writing of additionalinformation for the next picture. At step 4, flags are set based on thestart code. At step 5, the bit rate value, the VBV buffer size value,and the VBV delay value are changed. At step 6, counters areincremented. At step 7, multiplexed data is written in the data buffer102.

Hereinafter, the flags and counters of the data analysis control section506 are described before the details of the steps are described. PICSIZEis a counter indicating the size of a picture which is one unit forprocessing and is used for rate control and detection of the nextadditional information position. USER_COUNT is a counter which indicatesthe start position of user data. USERSIZE is a counter which indicatesthe size of the user data. PICDATA_COUNT is a counter which indicatesthe start position of picture data. These four counters respectivelycorrespond to the picture size included in the picture information 301,the user data position 302, the user data size 303, and the picture dataposition 304 shown in FIG. 3. SEQHEAD_FLAG, GOPHEAD_FLAG, PICHEAD_FLAG,USER_FLAG and SLICE_FLAG are flags which respectively indicate that thestart codes of the sequence header, the GOP header, the picture header,the user data, and the slice have been detected. PICTOP_COUNT is acounter which indicates the number of bytes in the picture header and isused for changing the VBV delay value. BP and WP are pointers of thedata buffer 102. BP is the first pointer which indicates the writeposition of the additional information 300. WP is the second pointerwhich indicates the write position of other data.

<Step 1: Taking in Input Data>

The value of the third input register 503 is written in the fourth inputregister 504. Thereafter, values are sequentially written, and 1-bytedata of the input code 121 is written in the first input register 501.

<Step 2: Detecting Start Code>

The start code detection section 505 determines whether 4-byte data ofthe first to fourth input registers 501 to 504 is identical to any ofthe start codes or all the bytes are 0x00.

<Step 3: Writing Additional Information>

The entire process of step 3 is performed as described below only underany of the following conditions: (a) a sequence header is detected; (b)SEQHEAD_FLAG=0 and a GOP header is detected; and (c) SEQHEAD_FLAG=0,GOPHEAD_FLAG=0, and a picture header is detected.

The first operation at step 3 is a process of writing the additionalinformation 300 in a predetermined format in the data buffer 102.Herein, the values of PICSIZE, USER_COUNT, USERSIZE, and PICDATA_COUNTare written in address locations designated by first pointer BP.

Then, the first and second pointers BP and WP are updated. Specifically,the value of second pointer WP is assigned to first pointer BP, and thevalue of second pointer WP is increased by the size of the additionalinformation 300. Through this operation, the position of the additionalinformation of the next picture is written in first pointer BP, and thenext position of the additional information is written in second pointerWP.

Lastly, the flags and counters are initialized. Specifically, PICSIZE,USER_COUNT, USERSIZE, PICDATA_COUNT and PICTOP_COUNT are all initializedto 0, and SEQHEAD_FLAG, GOPHEAD_FLAG, PICHEAD_FLAG and USER_FLAG are allcleared to 0.

<Step 4: Flag Update Process>

According to a result of the start code detection, corresponding flagsare cleared and set. Specifically, (1) when a sequence header isdetected, flags are set such that SEQHEAD_FLAG=1, GOPHEAD_FLAG=0,PICHEAD_FLAG=0, and USER_FLAG=0. (2) When a GOP header is detected,flags are set such that GOPHEAD_FLAG=1, PICHEAD_FLAG=0 USER_FLAG=0. (3)When a picture header is detected, flags are set such thatPICHEAD_FLAG=1 and USER_FLAG=0. (4) When a start code of user data isdetected, the content of GOPHEAD_FLAG is set in USER_FLAG. (5) When astart code of a slice is detected, flags are set such thatSEQHEAD_FLAG=0, GOPHEAD_FLAG=0, PICHEAD_FLAG=0, USER_FLAG=0, andSLICE_FLAG=1.

<Step 5: Data Rewrite Process>

The bit rate value, the VBV buffer size value, and the VBV delay valueare changed as described above. When SEQHEAD_FLAG=1, PICSIZE indicatesthe number of bytes counted from the sequence header. The bit rate valueand the VBV buffer size value can be determined based on the value ofPICSIZE. Further, 2 bytes of the VBV delay value are determined usingPICTOP_COUNT and rewritten.

<Step 6: Incrementing Counter>

It is assumed that step 6 and step 7 are skipped when the values of thefirst to fourth input registers 501 to 504 are all 0x00. That is, step 7of data writing is skipped, and as a result, stuffed zeros (redundantdata included in the main data) are deleted. According to the standards,deletion of such a pattern does not cause any adverse effect.

If any of the first to fourth input registers 501 to 504 has a valueother than 0x00, the data analysis control section 506 operates asdescribed below. Specifically, PICSIZE is incremented irrespective ofthe flags. USER_COUNT is incremented only when USER_FLAG=0 andPICHEAD_FLAG=0. USERSIZE is incremented only when USER_FLAG=1.PICDATA_COUNT is incremented only when SLICE_FLAG=0. PICTOP_COUNT isincremented only when PICHEAD_FLAG=1. With these conditions, thecounters can count a predetermined size according to occurrence ofrespective start codes.

<Step 7: Writing Data>

If any of the first to fourth input registers 501 to 504 has a valueother than 0x00, the value of the fourth input register 504 is writtenin an address position in the data buffer 102 which is indicated bysecond pointer WP, and second pointer WP is increased by 1.

It should be noted that, as the write data size 124, the value of firstpointer BP of the data analysis control section 506 is output as it is.That is, the write data size 124 indicates an address in which theadditional information 300 is last written.

The format shown in FIG. 3 is realized by writing data in the databuffer 102 according to the flow as described above. The most importantpoint here is that the data buffer 102 is accessible in such a mannerthat the region of the GOP user data 307 is distinguished from theregion of the other main data. Another method may be employed so long assuch distinguishment is achieved. For example, a buffer may be providedas a separate region. However, the user efficiency of a buffer isimproved when data is distinguished in such a manner that the additionalinformation 300 is added onto a single buffer, rather than when aseparate buffer is prepared. Other information, such as the originaldata size, or the like, may be further added to the additionalinformation 300.

FIG. 6 shows an example of an internal structure of the multiplexingsection 103 shown in FIG. 1. The multiplexing section 103 of FIG. 6includes an additional information reading section 601 for sequentiallyoutputting additional information, a main additional information buffer602 for holding the additional information, a main data reading section603 for sequentially outputting main data, a user data reading section604 for sequentially outputting user data, a user additional informationbuffer 605 for holding additional information which is referred to bythe user data reading section 604, a multiplexation control section 606,and a main data reprocessing section 701. The main additionalinformation buffer 602 is used in multiplexation of main data such thatadditional information is deleted every time one picture is multiplexed.On the other hand, the user additional information buffer 605 is used inmultiplexation of user data such that additional information is deletedonly when the user data is successfully multiplexed. Reference numerals621 and 702 denote main data. Reference numerals 622 and 703 denote maindata valid signals. Reference numeral 623 denotes user data. Referencenumeral 624 denotes a user data valid signal. Reference numeral 625denotes a frame number. Reference numeral 704 denotes a size changeinstruction. The multiplexation control section 606 outputs the maindata 702, the user data 623, start codes, etc., at appropriate timings,thereby outputting an output code 127 which has the format shown in FIG.4. The write data size 124 indicates up to where the data analyzingsection 101 has written data, thereby preventing the multiplexingsection 103 from erroneously processing unwritten data. The functions ofthe main data reprocessing section 701 will be described later.

A general operation of the multiplexation control section 606 is nowdescribed. In the first place, the amount of user data which can beplaced in each picture layer is calculated based on the data of the mainadditional information buffer 602. Then, the sequence header 305, theGOP header 306 and the picture header 308 shown in FIG. 3 aresequentially output. Thereafter, the GOP user data 307 is multiplexed tothe amount of data which can be placed, and then, the picture data 309is output. As a result, the sequence header 401, the GOP header 402, thepicture header 403, the picture user data 404, and the picture data 405are obtained.

Herein, the data amount after translation is increased by the placementof the picture user data 404, but desired user data can necessarily beplaced within a plurality of frames. Since the amount of data which canbe placed is calculated in advance, this placement process does notcause the rate control to fail. Since the bit rate value is increasedfrom the original value, data which cannot be placed do not occur insuccession.

More detailed descriptions are now provided. The additional informationreading section 601 includes a read pointer and a counter for thepicture size. The additional information reading section 601 startsoperations when the write address 122 is greater than the read pointerand the main additional information buffer 602 has a vacant space. Inthe first place, the additional information reading section 601 readsthe additional information 300 from the data buffer 102 using the readpointer and writes the additional information 300 in the main additionalinformation buffer 602. Then, the position of the next additionalinformation is identified using picture size information obtained fromthe picture information 301, and the read pointer of the data buffer 102is increased to the identified position. The main additional informationbuffer 602 can store plural sets of additional information.

The main data reading section 603 sequentially reads only the main databased on the additional information stored in the main additionalinformation buffer 602 and outputs the read main data to the main datareprocessing section 701. Although details will be described later, themain data reprocessing section 701 generally supplies the given maindata 621 as it is to the multiplexation control section 606 as the maindata 702. After reading of one picture is completed, correspondingadditional information is deleted from the main additional informationbuffer 602. When effective main data 621 is prepared, the main datareading section 603 sets the main data valid signal 622 to 1 andnotifies the multiplexation control section 606 through the main datareprocessing section 701 that the effective main data 621 is prepared.In this case, the main data reprocessing section 701 supplies the givenmain data valid signal 622 as it is to the multiplexation controlsection 606 as the main data valid signal 703.

The user data reading section 604 sequentially writes the additionalinformation in the user additional information buffer 605 andsequentially performs only reading of the GOP user data 307 according tothe written additional information. In this process, if the size of theuser data is 0, the next picture is searched for. When effective userdata 623 is prepared, the user data reading section 604 sets the userdata valid signal 624 to 1 and outputs a frame number 625 whichcorresponds to the effective user data 623. The frame number 625includes information about how many pictures from the leading end apicture including this data is and information about how many words fromthe leading end the user data which is to be read next is. The framenumber 625 indicates information about how many frames the user datawhich is to be placed should be regarded as belonging to.

The multiplexation control section 606 waits for the main data validsignal 703 to change to 1 every single picture and operates as describedbelow. In the first place, the multiplexation control section 606determines whether or not user data is to be placed at a currentmultiplexation subject picture. That is, the multiplexation controlsection 606 determines an optimum picture at which the user data is tobe placed while performing a simulation such that the VBV buffer doesnot go out of operation.

When the user data valid signal 624 is 0, the user data is not placed.

When the user data valid signal 624 is 1 and the frame number 625 isequal to or smaller than the picture number of main data which is amultiplexation subject, the user data is placed at a currently-processedpicture as much as possible. Thus, a buffer simulation is performed onthe current picture on the assumption that the user data has beenplaced. If no failure occurs in the simulation, the user data isactually placed at the current picture. Specifically, the picture sizeachieved after the user data is placed is calculated from the picturesize included in the picture information 301 and the user data size 303obtained from the main additional information buffer 602, and it ischecked whether or not the currently-occupied area of the VBV buffer isgreater than the calculated picture size achieved after the placement ofthe user data, whereby it is determined whether or not a failure occursin the simulation.

When the user data valid signal 624 is 1 and the frame number 625 isgreater than the picture number of main data which is a multiplexationsubject, the process is repeated as long as it is possible whilechanging the position at which the user data is placed on a picture bypicture basis (for example, the process is first performed with the userdata placed at the current picture, and then, the process is performedagain with the user data placed at the next picture) till the process isperformed with the user data placed at a frame identified by the framenumber 625. These simulations are possible when the additionalinformation corresponding to the frames up to a pertinent frame arestored in the main additional information buffer 602. If additionalinformation of a picture corresponding to the frame number 625 is notstored in the main additional information buffer 602, the user data isnot placed.

If through the series of above processes the position where the VBVbuffer does not go out of operation is found to be only the currentmultiplexation subject picture, the user data is placed at the currentpicture.

Through the above determination of conditions, it is determined whetheror not the picture user data 404 is to be placed. If it is determinedthat the picture user data 404 is to be placed, the picture user data404 is placed next to the picture header 403 together with a user startcode.

It should be noted that, since the leading position of the picture data309 is indicated by the picture data position 304 included in theadditional information 300, multiplexation of the picture data 405 canbe achieved extremely readily. Lastly, the value of the VBV bufferoccupation amount is calculated based on the multiplexed data amount ofthe output code 127. This value is used in multiplexation of the nextpicture.

As described above, the multiplexing section 103 of FIG. 6 operates tomultiplex the picture user data 404 at a position indicated by the framenumber 625 so long as it is possible. That is, a synchronizationdifference between main data and user data in the output code 127 isminimized.

The multiplexing section 103 may operate such that, if the frame number625 is greater than the picture number of main data which is amultiplexation subject, buffer simulation is not performed, and thepicture user data 404 is not placed. In this case, the user data iswritten in a picture which is a subsequent processing unit. This methodis realized only with a one-picture area in the main additionalinformation buffer 602 and therefore simplifies the process.

As described above, according to this embodiment, in the case where thedata amount is increased in format conversion of only user data, theupper limit of the data amount is raised by, for example, increasing thebit rate value, while the lower limit of the data amount is lowered byswitching from CBR to VBR, and user data is inserted into the picturelayer to the amount of data which can be placed, whereby codetranslation is achieved. Since only the user data is subjected to theformat conversion, unnecessary processes regarding picture data areremoved. As a result, the code translation is achieved more quicklywhile no deterioration occurs in picture quality.

In the above-described operation, when the synchronization differencebetween main data and user data in the output code 127 exceeds apredetermined amount, the main data reprocessing section 701 operates tochange the data amount of the main data.

FIG. 7 shows an example of an internal structure of the main datareprocessing section 701 shown in FIG. 6. The main data reprocessingsection 701 of FIG. 7 includes an I-picture decoder 801, an I-pictureencoder 802, a selector 803 and a size control section 804. Only whenthe main data 621 received from the main data reading section 603 is anI-picture, if the size change instruction 704 indicates that thesynchronization difference between main data and user data in the outputcode 127 exceeds a predetermined amount, the main data reprocessingsection 701 decodes the main data 621 and re-encodes a result of thedecoding, thereby reducing the data amount of the main data 702. If thesize change instruction 704 is not given, the selector 803 selects themain data 621 and the main data valid signal 622 supplied from the maindata reading section 603, and the selected main data 621 and main datavalid signal 622 are supplied as they are to the multiplexation controlsection 606 as the main data 702 and main data valid signal 703,respectively.

Herein, especially problematic is that the user data cannot be insertedbecause of too much data amount of one frame. Such a case occursfrequently in I-pictures, from which other pictures are not referred to,as compared with P-pictures and B-pictures. Thus, the multiplexationcontrol section 606 monitors the picture type included in the pictureinformation 301. Only when it is found by a VBV buffer simulation thatinsertion of user data in an I-picture causes the buffer to go out ofoperation, the multiplexation control section 606 gives the size changeinstruction 704 to the size control section 804.

Receiving the size change instruction 704, the size control section 804gives a size designation 805 to the I-picture encoder 802. The I-pictureencoder 802 re-encodes a result of decoding by the I-picture decoder 801to generate main data which has a reduced data amount and output a maindata valid signal in substitution for the main data valid signal 622.According to a switch signal 806 supplied from the size control section804, the selector 803 supplies the main data and main data valid signalreceived from the I-picture encoder 802 to the multiplexation controlsection 606 as the main data 702 and main data valid signal 703,respectively.

It should be noted that the situation considered herein is not likely tooccur frequently. According to this embodiment, re-encoding is performedin the main data reprocessing section 701 only in the above-describedspecial case. Thus, deterioration in process speed is not frequentlycaused, and the effect of suppressing a synchronization difference isachieved.

It should be noted that the present invention is applicable not only toformat conversion of user data but also to addition of user data.

Industrial Applicability

As described above, in a code translation method and code translationdevice according to the present invention, format conversion andaddition of user data are realized without a failure of rate control.Therefore, the code translation method and code translation device areuseful in code translation of compressed/encoded multimedia information.

1. A code translation method comprising the steps of: receiving an input code stored in a hierarchical data structure, the input code including a first value in a parameter of the hierarchical data structure which determines an allowable range of data amount in the input code, user data in a first layer of the hierarchical data structure, and main data in a second layer of the hierarchical data structure; storing the user data and main data in a data buffer; and generating an output code stored in the hierarchical data structure, the output code including the stored main data, the stored user data in a layer of the hierarchical data structure other than the first layer, and a second value in the parameter of the hierarchical data structure which determines an allowable range of data amount in the output code, the second value being different from the first value to reflect a change from the allowable range of data amount in the input code effected by moving the user data from the input code to the output code; wherein the stored main data included in the output code is identical to the main data included in the input code.
 2. The code translation method of claim 1, wherein the hierarchical data structure conforms with the ISO 13818-2 standard; the parameter of the hierarchical data structure is one of a bit rate value, a VBV (Video Buffering Verifier) buffer size value, and a VBV delay value; and the main data comprises compressed video data.
 3. The code translation method of claim 2, wherein the first layer of the hierarchical data structure is the Group of Pictures (GOP) layer; and the third layer of the hierarchical data structure is the picture layer.
 4. The code translation method of claim 2, wherein the first layer of the hierarchical data structure is the picture layer; and the third layer of the hierarchical data structure is the Group of Pictures (GOP) layer.
 5. The code translation method of claim 1, further comprising the step of generating additional information for distinguishing the user data included in the input code from the main data, wherein generation of the output code is advanced according to the additional information.
 6. The code translation method of claim 1, wherein the stored main data included in the output code is in the second layer of the of the hierarchical data structure.
 7. A code translation device comprising: a data analyzing section adapted to identify in an input code stored in a hierarchical data structure a first value in a parameter of the hierarchical data structure which determines an allowable range of data amount in the input code, user data in a first layer of the hierarchical data structure, and main data in a second layer of the hierarchical data structure; and a multiplexing section which produces an output code stored in a hierarchical data structure, the output code including the stored main data, the user data in a layer of the hierarchical data structure other than the first layer, a second value in the parameter of the hierarchical data structure which determines an allowable range of data amount in the output code, the second value being different from the first value to reflect a change from the allowable range of data amount in the input code effected by moving the user data from the input code to the output code; wherein the main data included in the output code is identical to the main data included in the input code.
 8. The code translation device of claim 7, wherein the hierarchical data structure conforms with the ISO 13818-2 standard; the parameter of the hierarchical data structure is one of a bit rate value, a VBV (Video Buffering Verifier) buffer size value, and a VBV delay value; and the main data comprises compressed video data.
 9. The code translation device of claim 8, wherein the first layer of the hierarchical data structure is the Group of Pictures (GOP) layer; and the third layer of the hierarchical data structure is the picture layer.
 10. The code translation device claim 8, wherein the first layer of the hierarchical data structure is the picture layer; and the third layer of the hierarchical data structure is the Group of Pictures (GOP) layer.
 11. The code translation device of claim 8, wherein the stored main data included in the output code is in the second layer of the of the hierarchical data structure. 